Lab 09 - Vision based SLAM: ORB-SLAM3

Robotics II

Poznan University of Technology, Institute of Robotics and Machine Intelligence

Laboratory 9: ORB-SLAM3 - Simultaneous localization and mapping using camera sensor

During the course, we perform visual SLAM (Simultaneous localization and mapping) for the racecar from the Formula Student Driverless Simulator. This time, we want to show you ORB-SLAM3 - high-speed, robust, and accurate trajectory estimation in many environments. It is currently one of the most robust and efficient vision SLAM systems.

The full code of ORB-SLAM3 with instructions is available on Github. During this class, the prepared Docker container with a full ORB_SLAM3 environment will be used.

Step 0: (do this if `arm/image04` image not exist in `docker images` list)

download prepared docker image:

wget  "https://chmura.put.poznan.pl/s/T98vgBxYTNZqgHO/download" -O arm.tar.gz

load into docker register

docker load < arm.tar.gz

add X11 to root group

xhost +local:root

Step 1: Run container with prepared ORB_SLAM3 environment

docker run \
  -it --gpus all --privileged --network=host \
  --env="DISPLAY=$DISPLAY" \
  --env="QT_X11_NO_MITSHM=1" \
  --env="NVIDIA_DISABLE_REQUIRE=1" \
  --env="XAUTHORITY=$XAUTH" \
  --env="NVIDIA_VISIBLE_DEVICES=all" \
  --env="NVIDIA_DRIVER_CAPABILITIES=all" \
  --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
  --volume="$XAUTH:$XAUTH" \
  --name=visual_slam \
  arm/image04

install your favorite text editor
```
apt update && apt install vim nano
```
run ROS master in the background
```
roscore &
```

download racecar rosbag

wget "https://chmura.put.poznan.pl/s/YfYfE2CQJ8fszWq/download" -O vision_slam.bag

check rosbag topics:
```
rosbag info vision_slam.bag
```

Step 2: Prepare algorithm config

camera topic info --- header: seq: 291 stamp: secs: 1645569572 nsecs: 157031000 frame_id: <HIDDEN> height: 720 width: 1280 encoding: "bgr8" is_bigendian: 0 step: 3840 data: "<array type: uint8, length: 2764800>" ---

TASK

Download the ORB_SLAM3 config:

wget "https://raw.githubusercontent.com/PUTvision/RoboticsII-FSDS/orb-slam3/FSDS.yaml"

Based on the information below and your knowledge, fill in the missing values. Tips:
* <WIDTH> - integer, the width of an image * <HEIGHT> - integer, the height of an image * <FX> - float, the focal length in pixel-related units (here <FX> and <FY> are the same); in this case 530.0 * <CX> and <CY> - floats, are principal points (that are usually at the image center) * <BF> - float, the multiplication of the camera base (the dimension between cameras, in this case base is 0.2 meters) and <FX>.

Step 3: Run the ORB_SLAM3 alghoritm:

rosrun ORB_SLAM3 Stereo /ORB_SLAM3/Vocabulary/ORBvoc.txt FSDS.yaml true <camera_topic_left> <camera_topic__right>

Simulteneus replay the rosbag. Use -r 0.25 for slowing down playback (it is related to the limitation of resources).

rosbag play -r 0.25 vision_slam.bag

As a result, upload the plot and screenshot from the ORB_SLAM3 GUI to the eKursy platform.